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Enhanced  Training  Could  Strengthen  DOD’s  Best 
Value  Tradeoff  Decisions 


Why  GAO  Did  This  Study 

The  Department  of  Defense  (DOD) 
obligated  about  $380  billion  in  fiscal 
year  2009  to  acquire  products  and 
services.  One  approach  DOD  can  take 
to  evaluate  offerors’  proposals  is  the 
best  value  tradeoff  process  in  which 
the  relative  importance  of  price 
varies  compared  to  non-cost  factors. 

The  National  Defense  Authorization 
Act  for  Fiscal  Year  2010  required 
GAO  to  review  DOD’s  use  of  the  best 
value  tradeoff  process,  specifically 
when  non-cost  factors  were  more 
important  than  price.  In  response, 
GAO  determined  (1)  how  often  and 
for  what  types  of  contracts  DOD  used 
the  best  value  tradeoff  process;  (2) 
why  and  how  DOD  used  such  an 
approach;  and  (3)  challenges,  if  any, 
DOD  faces  in  using  the  best  value 
tradeoff  process. 

GAO  identified  a  probability  sample 
of  new,  competitively  awarded  fiscal 
year  2009  contracts  in  which  DOD 
obligated  $25  million  or  more.  GAO 
reviewed  guidance,  solicitations, 
source  selection  decisions,  and  other 
documents  for  129  contracts  and 
interviewed  DOD  contracting  and 
program  staff  about  the  use  of  the 
best  value  tradeoff  process. 

What  GAO  Recommends 

GAO  recommends  that  to  help  DOD 
effectively  employ  best  value  tradeoff 
processes,  DOD  develop  training 
elements,  such  as  case  studies,  that 
focus  on  reaching  tradeoff  decisions, 
as  it  updates  its  training  curriculum. 
DOD  concurred  with  this 
recommendation. 


View  GAO-1 1-8  or  key  components. 

For  more  information,  contact  John  Hutton  at 
(202)  512-4841  or  huttonj@gao.gov. 


What  GAO  Found 

In  fiscal  year  2009,  DOD  used  best  value  processes  for  approximately 
95  percent  of  its  new,  competitively  awarded  contracts  in  which  $25  million  or 
more  was  obligated.  Almost  half  of  DOD’s  contracts — 47  percent — ^were 
awarded  using  a  tradeoff  process  in  which  non-cost  evaluation  factors,  when 
combined,  were  more  important  than  price.  DOD  used  best  value  tradeoffs 
principally  to  acquire  services,  such  as  construction  of  troop  housing,  as  well 
as  for  professional  management  services. 

DOD  used  the  best  value  tradeoff  process  in  88  of  the  129  contracts  GAO 
reviewed.  For  60  of  the  88  contracts,  DOD  weighted  non-cost  factors  as  more 
important  than  price.  In  these  cases,  DOD  was  willing  to  pay  more  for  a 
contractor  that  demonstrated  it  understood  complex  technical  issues  more 
thoroughly,  could  provide  a  needed  good  or  service  to  meet  deadlines,  or  had 
a  proven  track  record  in  successfully  delivering  products  or  services  of  a 
similar  nature.  In  making  tradeoff  decisions,  GAO  foimd  that  DOD  selected  a 
lower  priced  proposal  nearly  as  often  as  it  selected  a  higher  technically  rated, 
but  more  costly  proposal.  Overall,  GAO  found  that  DOD  paid  a  combined  total 
of  more  than  $230  million  in  price  differentials — the  difference  in  price 
between  the  awardee  and  the  offeror  next  in  line  for  award — on  21  contracts, 
but  chose  not  to  pay  more  than  $800  million  in  proposed  costs  by  selecting  a 
lower  priced  offer  over  a  higher  technically  rated  offer  in  18  contracts.  DOD 
does  not  track  whether  the  use  of  best  value  tradeoff  processes  correlates 
with  the  contractor  successfully  meeting  the  terms  of  the  contract  and  noted 
that  many  factors  ultimately  contribute  to  an  acquisition’s  success  or  failure. 

DOD  officials  identified  several  challenges  in  using  the  best  value  tradeoff 
process,  including  the  difficulty  in  determining  meaningful  evaluation  factors 
and  the  business  judgment  of  acquisition  staff  required.  DOD  officials  also 
noted  that  the  complexity  of  the  tradeoff  process  increases  the  risk  of  bid 
protests.  For  example,  GAO  foimd  that  15  of  the  88  contracts  awarded  using  a 
best  value  tradeoff  process  reviewed  were  protested  to  GAO,  resulting  in 
4  cases  in  which  DOD  terminated  the  contract  or  made  a  new  source  selection 
decision  when  DOD  determined  that  it  failed  to  adhere  to  the  solicitations’ 
requirements.  Such  concerns  are  heightened  given  the  expected  influx  of 
more  than  6,400  new  contracting  personnel  over  the  next  few  years. 

According  to  DOD  officials,  making  sound  tradeoff  decisions,  and  in 
particular,  deciding  whether  or  not  a  price  differential  is  warranted,  is  one  of 
the  most  difficult  aspects  of  using  a  best  value  tradeoff  process.  DOD  is 
developing  a  new  departmentwide  source  selection  guide  and  intends  to 
subsequently  revise  its  training  curriculum,  but  neither  the  guide  nor  DOD’s 
current  training  curriculum  provides  agency  personnel  with  information  on 
assessing  price  differentials  when  performing  tradeoff  analyses. 
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Indefinite  Delivery  Indefinite  Quantity 

LPTA 

Lowest  Price  Technically  Acceptable 

This  is  a  work  of  the  U.S.  government  and  is  not  subject  to  copyright  protection  in  the 

United  States 

The  pubiished  product  may  be  reproduced  and  distributed  in  its  entirety 

without  further  permission  from  GAO.  However,  because  this  work  may  contain 
copyrighted  images  or  other  materiai,  permission  from  the  copyright  hoider  may  be 
necessary  if  you  wish  to  reproduce  this  materiai  separateiy. 

Page  ii 


GAO-11-8  Best  Value  Contracting 


GAO 

^^^^^^^—^Accountability  *  Integrity  *  Reliability 

United  States  Government  Acconntability  Office 
Washington,  DC  20548 


October  28,  2010 

The  Honorable  Carl  Levin 
Chairman 

The  Honorable  John  McCain 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Ike  Skelton 
Chairman 

The  Honorable  Howard  P.  McKeon 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Department  of  Defense  (DOD)  obligated  about  $380  billion  in  fiscal 
year  2009^  to  acquire  products  and  services  needed  to  support  its 
missions.  DOD  has  several  approaches  it  can  use  to  evaluate  offerors’ 
proposals.  For  example,  DOD  can  use  a  best  value  tradeoff  process,  in 
which  it  can  vary  the  relative  importance  of  cost  or  price  to  other  factors, 
such  as  technical  approach  or  past  performance  in  its  solicitations  to 
offerors.  In  these  cases,  DOD  may  award  a  contract  to  other  than  the 
lowest-priced  offeror  if  DOD  determines  that  a  higher-priced  proposal 
provides  a  greater  benefit  to  DOD,  and  this  greater  benefit  is  worth  paying 
an  additional  cost,  or  price  differential.  Proper  execution  of  the  tradeoff 
process  is  essential,  however,  to  ensure  that  DOD  lays  the  foundation  for 
successful  acquisition  outcomes.  We  have  identified  DOD  contract 
management  as  a  long-standing  high-risk  area,  due  in  part  to  DOD’s  use  of 
ill-suited  business  arrangements  that  have  contributed  to  unmet 
expectations  and  placed  the  department  at  risk  of  potentially  paying  more 
than  necessary.^ 


^  According  to  the  Federal  Procurement  Data  System-Next  Generation,  DOD’s  total 
obligations  in  fiscal  year  2009  were  about  $366  billion.  However,  this  figure  reflects  an 
approximately  $13.9  billion  downward  adjustment  made  by  DOD  to  correct  an 
administrative  error  made  in  fiscal  yecu:  2008.  As  this  adjustment  significantly  affected 
DOD’s  reported  obligations  in  fiscal  year  2009,  the  $380  billion  figure  we  report  reflects 
what  DOD’s  total  obligations  would  have  been  had  the  error  not  occurred. 

^  GAO,  High-Risk  Series:  An  Update,  GAO-09-271  (Washington,  D.C.:  January  2009). 
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Section  845  of  the  National  Defense  Authorization  Act  for  Fiscal  Year  2010 
directed  GAO  to  report  on  DOD’s  use  of  the  best  value  tradeoff  process, 
and  specifically  for  cases  in  which  DOD  determined  that  it  would  evaluate 
contractors’  proposals  on  factors  other  than  cost  or  price,  if  these  non-cost 
factors,  when  combined,  were  to  be  considered  more  important  than  cost 
or  price.  ^  To  respond  to  the  mandate,  we  determined  (1)  how  often  and  for 
what  types  of  contracts  DOD  used  the  best  value  tradeoff  process;  (2)  why 
and  how  DOD  used  the  best  value  tradeoff  process;  and  (3)  what 
challenges,  if  any,  DOD  faces  in  using  the  best  value  tradeoff  process. 

To  conduct  our  work  we  used  the  Federal  Procurement  Data  System-Next 
Generation  (FPDS-NG)'‘  to  identify  new,  competitively  awarded  contracts 
in  which  DOD  obligated  $25  million  or  more  in  fiscal  year  2009.  We 
selected  the  $25  million  threshold  based  on  a  Defense  Federal  Acquisition 
Regulation  Supplement  (DFARS)  requirement  that  contracts  for 
production  or  services  with  $25  million  or  more  in  estimated  total  costs  for 
any  fiscal  year  have  written  acquisition  plans,  which  contain  information 
on  the  anticipated  source  selection  approach.^  This  analysis  identified  363 
contracts  on  which  DOD  had  obligated  a  total  of  $39.2  billion,  or  about  10 
percent  of  the  total  amoimt  DOD  obligated  on  contracts  in  fiscal  year  2009. 
From  this  population,  we  selected  a  random  sample  of  160  contracts, 
including  60  indefinite  delivery  contracts.®  We  identified  errors  in  the 
information  provided  by  FPDS-NG  on  31  contracts,  including  contracts 
that  were  incorrectly  coded  as  competitively  awarded  or  had  incorrect 


®  Pub.  L.  No.  111-84  (2009). 

^  The  Federal  Procurement  Data  System-Next  Generation  is  the  federal  government’s 
system  for  tracking  information  on  contracting  actions. 

®  DFARS  207.103(d)Ci)(B). 

®  Indefinite  delivery  contracts  can  be  used  when  the  government  needs  flexibility  in  the 
timing  of  orders  within  a  specified  period  of  time.  One  type  of  indefinite  delivery  contract 
is  an  indefinite  delivery/indefinite  quantity  (IDIQ),  which  allows  the  government  to  order 
unspecified  quantities,  within  stated  limits,  of  products  or  services  during  a  fixed  period 
when  they  cannot  predetermine  their  needs.  After  award  of  the  base  IDIQ  contract, 
products  and  services  are  procured  through  individual  delivery  or  task  orders  during  the 
contract  period  based  on  governmental  needs.  IDIQ  contracts  may  be  issued  ais  a  single 
award  to  one  contractor,  or  preferably  on  a  multiple  award  basis  to  several  contractors,  in 
which  case  the  FAR  requires  that  each  awardee  be  given  a  fair  opportunity  to  compete  for 
subsequent  orders. 
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award  amounts.^  We  excluded  these  contracts  from  our  sample  and 
determined  that  FPDS-NG  was  sufficiently  rehable  for  the  purposes  of  our 
review  after  adjusting  for  these  errors.  Based  on  our  analysis  of  the 
remaining  129  sample  contracts,  we  produced  estimates  of  source 
selection  approaches  used,  contract  type,  and  whether  a  product  or 
service  was  acquired.  To  determine  the  type  of  and  rationale  for  the 
source  selection  process  used  for  the  contracts,  we  reviewed  the 
associated  acquisition  plans,  solicitations,  source  selection  decision 
memorandums,  and  other  relevant  documents  for  each  of  the  contracts  in 
our  sample.  In  particular,  we  reviewed  source  selection  decision 
documents  for  contracts  that  used  a  tradeoff  process  to  determine 
whether  DOD  paid  a  price  differential.  For  the  purposes  of  this  report,  we 
defined  a  price  differential  as  the  difference  in  price  between  the 
awardee’s  price  and  the  price  of  the  offeror  next  in  line  for  award. 

From  our  sample,  we  also  judgmentally  selected  buying  activities  from 
each  military  department  and  one  defense  agency  based  on  such  factors  as 
the  number  of  best  value  contracts,  the  type  of  contract,  and  the  type  of 
product  or  service  acquired.  Results  based  on  these  selected  buying 
activities  are  not  generalizable  to  a  larger  population.  At  each  activity,  we 
reviewed  contract  files  and  interviewed  program  and  contracting  officials 
to  discuss  their  rationale  for  selecting  a  tradeoff  process  including  the 
selection  of  non-cost  evaluation  factors  for  34  contracts  and  23  task  orders 
as  illustrative  case  studies.  We  also  reviewed  contract  documents,  DOD 
and  military  department  source  selection  guidance,  and  interviewed 
program  and  contracting  officials  from  DOD,  the  military  departments, 
and  one  defense  agency  to  identify  what  challenges,  if  any,  DOD  faces  in 
using  the  best  value  tradeoff  process.  A  more  detailed  description  of  our 
scope  and  methodology  is  included  in  appendix  1. 

We  conducted  this  performance  audit  from  March  2010  through  October 
2010  in  accordance  with  generally  accepted  government  auditing 
standards.  Those  standards  require  that  we  plan  and  perform  the  audit  to 
obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe 


’’  We  have  previously  reported  on  data  reliability  issues  with  FPDS-NG.  See,  e.g.,  GAO, 
Federal  Contracting:  Observations  on  the  Government’s  Contracting  Data  Systems, 
GAO-09-1032T  (Washington,  D.C.:  Sept.  29,  2009);  and  Contract  Management:  Minimal 
Compliance  with  New  Safeguards  for  Time-and-Materials  Contracts  for  Commercial 
Services  and  Safeguards  Have  Not  Been  Applied  to  GSA  Schedules  Program,  GAO-09-579 
(Washington,  D.C.:  June  24,  2009). 
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that  the  evidence  obtained  provides  a  reasonable  basis  for  our  findings 
and  conclusions  based  on  our  audit  objectives. 


Background 


The  Federal  Acquisition  Regulation  (FAR)  Part  15  allows  the  use  of  several 
best  value  competitive  source  selection  techniques  to  meet  agency  needs. 
Within  the  best  value  continuum,  DOD  may  choose  an  approach  that  it 
considers  the  most  advantageous  to  the  government,  including  the  lowest 
price  technically  acceptable  (LPTA)  process  and  the  tradeoff  process. 

DOD  may  elect  to  use  the  LPTA  process  in  acquisitions  where  the 
requirement  is  clearly  definable  and  the  risk  of  unsuccessful  contract 
performance  is  minimal.  In  such  cases,  DOD  may  determine  that  cost  or 
price  should  play  a  dominant  role  in  source  selection.  When  using  the 
LPTA  process,  DOD  specifies  its  minimal  technical  requirements  in  the 
solicitation.  Once  DOD  determines  that  the  contractors  meet  or  exceed 
the  technical  requirements,  no  tradeoffs  between  cost  or  price  and  non¬ 
cost  factors  are  permitted  and  the  award  is  made  based  on  the  lowest 
price  offered  to  the  government. 

By  contrast,  DOD  may  elect  to  use  a  tradeoff  process  in  acquisitions  where 
the  requirement  is  less  definitive,  more  development  work  is  required,  or 
the  acquisition  has  greater  performance  risk.  In  these  instances,  non-cost 
evaluation  factors,  such  as  technical  capabilities  or  past  performance,  may 
play  a  dominant  role  in  the  source  selection  and  tradeoffs  among  price  and 
non-cost  factors  allow  DOD  to  accept  other  than  the  lowest  priced 
proposal.  This  report  focuses  on  DOD’s  use  of  the  tradeoff  process,  and 
specifically,  in  which  non-cost  factors,  when  combined,  were  considered 
more  important  than  cost  or  price. 

When  using  a  tradeoff  process,  the  FAR  requires  that  evaluation  factors 
and  significant  subfactors  that  affect  contract  award  and  their  relative 
importance  be  clearly  stated  in  the  solicitation;  and  the  solicitation  must 
provide  whether  all  evaluation  factors  other  than  cost  or  price,  when 
combined,  are  significantly  more  important  than,  approximately  equal  to, 
or  significantly  less  important  than  cost  or  price.®  Additionally,  the  FAR 
requires  that  each  factor  represent  key  areas  of  importance  and  emphasis 


®  Hereafter  for  the  purposes  of  this  report,  we  use  the  term  “non-cost  factors”  for  both  non¬ 
cost  and  non-price  evaluation  factors  and  the  term  “price”  for  both  cost  and  price,  despite 
the  different  ways  that  they  are  evaluated  under  the  FAR. 
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to  be  considered  in  the  source  selection  decision  and  that  support 
meaningful  comparison  and  discrimination  between  and  among  competing 
proposals.”  The  FAR  also  requires  the  source  selection  authority 
document  the  perceived  benefits  of  the  higher  priced  proposal  and  the 
rationale  for  tradeoffs  in  the  contract  file.“  The  resulting  source  selection 
decision  should  be  based  on  a  comparative  assessment  of  proposals 
against  all  source  selection  criteria  in  the  solicitation.  The  decision  must 
also  include  the  rationale  for  any  business  judgments  and  tradeoffs  made 
or  relied  on  by  the  source  selection  official,  including  benefits  associated 
with  additional  costs.  Although  the  rationale  for  the  source  selection 
decision  must  be  documented,  the  documentation  need  not  quantify  the 
tradeoffs  that  led  to  the  decision. 

In  fiscal  year  2009,  DOD  obligated  about  $380  billion  on  contracts  for 
goods  and  services.  Our  analysis  of  data  reported  by  DOD  to  FPDS-NG 
indicates  that  $69.9  billion  or  18  percent  of  DOD’s  obligations  were  made 
on  new  contracts  competitively  awarded  in  fiscal  year  2009  (see  figure  1). 
By  contrast,  about  $176  billion  were  modifications  to  or  orders  issued 
under  contracts  that  were  awarded  prior  to  fiscal  year  2009  and  $133 
billion  were  awarded  non-competitively,“  which  in  combination  totaled  to 
nearly  82  percent  of  DOD’s  reported  contract  obligations  in  fiscal  year 
2009. 


”  FAR  16.304(b). 

“  FAR  15.308. 

For  additional  information  see  GAO,  Federal  Contracting:  Opportunities  Exist  to 
Increase  Competition  and  Assess  Reasons  When  Only  One  Offer  is  Received,  GAO-10-833 
(Washington  D.C.:  July  26,  2010). 
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Figure  1 :  DOD  Contract  Obligations  in  Fiscal  Year  2009°  (Dollars  in  billions) 


N on-competed  contracts*’ 


More  than  $25  million 
obligated 


Less  than  $25  million 
obligated 


More  than  $25  million 
obligated 


Less  than  $25  million 
obligated 


Competed  contracts 

New  contracts 

Orders  and  modifications  on  contracts 

_  awarded  prior  to  FY  2009 

Source:  GAO  analysis  of  FPDS-NG  obligations  data 

“Dollar  amounts  may  not  sum  due  to  rounding. 

“The  non-competed  contracts  category  consists  of  those  contracts  awarded  under  other  than  full  and 
open  competition  as  defined  In  FAR  subpart  6.3. 

Properly  managing  the  acquisition  of  goods  and  services  requires  an 
acquisition  workforce  with  the  right  skills  and  capabilities.  In  March  2009, 
however,  we  reported  that  DOD  lacked  complete  information  on  the  skill 
sets  of  the  current  acquisition  workforce  and  whether  these  skill  sets  were 
sufficient  to  accomplish  its  missions.  In  April  2009,  the  Secretary  of 
Defense  announced  his  intent  to  grow  the  acquisition  workforce  by  15 
percent  by  fiscal  year  2015.  As  part  of  this  strategy,  DOD  indicated  that  it 
intends  to  grow  its  contracting  career  field  by  more  than  6,400  personnel, 
an  increase  of  more  than  28  percent  from  fiscal  year  2008  staffing  levels. 


GAO,  Department  of  Defense:  Additional  Actions  and  Data  Are  Needed  to  Effectively 
Manage  and  Oversee  DOD’s  Acquisition  Workforce,  GAO-09-342  (Washington  D.C.:  March 
25,  2009). 
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DOD  Relies  Heavily 
on  Best  Value 
Processes  to  Evaluate 
Contractor  Offers 


DOD  relies  heavily  on  the  use  of  the  best  value  process  to  evaluate  offers 
from  potential  contractors.  DOD  chose  a  best  value  process  for 
approximately  95  percent  of  its  new,  competitively  awarded  contracts  on 
which  it  had  obligated  $25  million  or  more  in  fiscal  year  2009.  “  Almost  half 
of  DOD’s  contracts — 47  percent — were  awarded  using  a  tradeoff  process 
in  which  non-cost  evaluation  factors,  when  combined,  were  more 
important  than  price.  Figure  2  shows  how  often  DOD  used  the  different 
best  value  processes  and  other  source  selection  approaches. 


Figure  2:  Estimated  Frequencies  of  Source  Selection  Approaches  Used  in  Fiscal 
Year  2009  for  New,  Competitively  Awarded  DOD  Contracts  Obligating  over 
$25  million’ 


Lowest  price 
technically  acceptable 

Sealed  bid‘= 

2% 

Non-cost  factors  less 
important  than  price 


Non-cost  factors 
equal  to  price 


Non-cost  factors  more 
important  than  price 


Best  value  processes  (95%) 

Best  value  tradeoff  process  (69%) 


Source:  GAO  analysis  of  DOD  contract  documents 


“The  95  percent  confidence  intervals  for  estimates  in  this  graph  are  within  +!-  8  percentage  points  of 
the  estimates  themselves. 

hn  sealed  bidding,  award  is  made  to  the  responsible  bidder  whose  bid  conforms  to  the  invitation  for 
bid  and  is  the  most  advantageous  for  the  government  considering  only  price  and  price-related  factors 
included  in  the  invitation. 


All  percentage  and  fraction  estimates  reported  within  this  section  have  95  percent 
confidence  intervals  within  plus  or  minus  8  percentage  points  of  the  estimates  themselves. 
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In  69  percent  of  the  contract  awards,  DOD  used  the  best  value  tradeoff 
process.  When  doing  so,  it  acquired  services  approximately  four  times  as 
often  as  it  acquired  products.  Over  half  of  these  procurements  were  for 
building  or  civil  engineering  construction  services,  including  projects  for 
troop  housing,  administrative  facilities,  and  hurricane  protection  systems. 
Other  services  procured  using  the  tradeoff  process  were  equipment 
maintenance  and  professional  management  services.  For  example,  in  fiscal 
year  2009,  the  Army  Corps  of  Engineers  awarded  a  contract  worth  more 
than  $963  million  to  construct  one  of  the  largest  pumping  stations  in  the 
world,  along  with  floodgates  and  floodwalls  for  hurricane  protection. 
Similarly,  the  Air  Force  awarded  the  Contract  Field  Team  program 
multiple-award  contract,  with  an  estimated  base  value  of  $2.6  billion  for 
modification,  maintenance  and  repair  of  systems  including  aircraft  and 
missile  defense  for  the  departments  of  the  Army,  Navy,  Air  Force  and 
several  federal  agencies.  Small  arms  and  electronic  countermeasure 
equipment  were  among  the  products  most  frequently  procured  using  a 
tradeoff  process,  including  the  contracts  for  the  Squad  Automatic 
Weapons — lightweight,  automatic  rifles  issued  to  each  Army  and  Marine 
rifle  squad — and  an  Army  contract  to  procure  devices  that  counteract 
radio-controlled  improvised  explosives.  Our  analysis  of  selected 
characteristics  of  contracts  awarded  using  a  best  value  tradeoff  process  in 
fiscal  year  2009  is  shown  in  figure  3. 
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Figure  3:  Estimated  Frequencies  of  Selected  Characteristics  of  Contracts  or  Agreements  Awarded  using  a  Best  Value 
Tradeoff  Process^ 


17% 


Service 


Source:  GAO  analysis  of  DOD  contract  documents  and  data  obtained  from  FPDS-NG. 

"Note:  The  95  percent  confidence  intervals  for  estimates  in  this  graph  are  within  +/-  8  percentage 
points  ofthe  estimates  themselves. 

“Percentages  may  not  sum  to  100  percent  due  to  rounding. 

“Combination  refers  to  contracts  thatallow  fororders  to  be  placed  using  more  than  one  pricing 
arrangement 

“Stand-alone  refers  to  contracts  thatdo  notallow  for  individual  orders  to  be  placed  againstthe 
contract 

'Other  includes  basic  ordering  agreements  and  blanket  purchase  agreements. 
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As  part  of  our  work,  we  reviewed  10 IDIQ  contracts  that  had  been 
awarded  using  a  best  value  tradeoff  process  and  23  task  and  delivery 
orders  under  these  contracts  which  had  obligations  ranging  from 
$11  million  to  over  $319  million.  In  most  cases,  DOD  did  not  issue  the  task 
or  delivery  orders  we  reviewed  using  a  tradeoff  process.  For  example,  an 
Air  Force  official  explained  that  the  initial  task  orders  for  the  Contract 
Field  Team  program,  including  13  orders  in  our  sample,  were  issued  under 
an  LPTA  process  because  data  needed  to  assess  contractor  performance 
and  timeliness  were  not  yet  available  to  use  given  the  short  time  between 
award  of  the  base  contract  and  issuance  of  the  first  orders.  An  Air  Force 
official  indicated  that  once  they  had  obtained  sufficient  performance  data, 
they  intended  to  issue  task  orders  using  a  tradeoff  process  when  possible. 
DOD  officials  issued  six  other  task  orders  on  the  basis  of  negotiating  with 
a  contractor  who  had  been  awarded  a  single  award  IDIQ  contract.  The 
four  remaining  orders  were  awarded  using  a  tradeoff  process.  For 
example,  the  Army  wanted  infrared  vision  enhancement  equipment  for 
nighttime  and  battlefield  use  in  Iraq  and  Afghanistan  to  be  delivered  as 
quickly  as  possible.  Consequently,  the  Army  used  a  contractor’s  ability  to 
meet  delivery  requirements  as  the  principal  evaluation  factor  in  selecting 
the  contractor  for  delivery  order  award. 

Some  DOD  officials  noted  that  the  use  of  various  source  selection 
evaluation  methods  can  change  over  time.  For  example: 

•  Defense  Logistics  Agency  (DLA)  officials  noted  that  they  have  recently 
transitioned  from  principally  using  the  tradeoff  process  to  using  the  LPTA 
process  for  most  fuel  purchases  because  the  majority  of  their 
procurements  were  for  a  commercial  product  in  relatively  stable  domestic 
and  international  markets.  They  noted,  however,  that  they  still  use  the 
tradeoff  process  in  less  stable  areas,  such  as  Iraq  and  Afghanistan,  where 
they  require  more  information  about  vendors’  past  performance  and 
technical  capability  when  operating  in  war  zones. 

•  Conversely,  Army  Corps  officials  in  New  Orleans  reported  that  they  have 
been  using  the  tradeoff  process  more  frequently  since  the  increase  in  civil 
works  construction  projects  following  Hurricane  Katrina.  While  they 
typically  used  sealed  bids  in  the  past,  they  told  us  that  use  of  the  tradeoff 
process  enabled  them  to  better  assess  contractors’  ability  to  meet  safety 
and  schedule  requirements. 
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DOD  Used  a  Best 
Value  Tradeoff 
Process  to  Address 
Complex  or  Time 
Sensitive  Needs,  but 
Paid  Relatively  Few 
Price  Differentials 


DOD  officials  tended  to  use  a  best  value  tradeoff  process  with  non-cost 
factors  weighted  more  important  than  price  when  they  were  willing  to 
accept  a  higher  price  if  a  contractor  could  demonstrate  certain 
advantages,  such  as  meet  a  deadline,  demonstrate  that  it  understood 
complex  technical  issues,  or  propose  an  innovative  approach.  DOD  often 
indicated  in  tradeoff  solicitations  that  non-cost  factors  would  be 
significantly  more  important  than  price  in  making  award  decisions,  but  our 
analysis  indicated  that  DOD  selected  a  lower  priced  proposal  among  those 
offerors  remaining  in  the  final  competition  almost  as  often  as  it  selected  a 
higher  technically  rated,  but  more  costly,  proposal.  Overall,  DOD  paid  a 
price  differential — the  difference  in  the  price  of  the  offeror  awarded  the 
contract  and  the  price  of  the  offeror  next  in  line  for  award — in  21  of  the  68 
contracts  in  which  a  price  differential  was  considered.  Most  differentials 
were  less  than  5  percent.  While  DOD  officials  told  us  that  the  tradeoff 
process  provides  an  essential  tool  to  obtain  desired  capabilities,  they  rely 
on  the  case-by-case  judgment  of  contracting  and  program  officials  to 
determine  the  best  acquisition  approach  suited  to  program  requirements 
and  do  not  specifically  track  whether  use  of  the  tradeoff  process  is  in 
DOD’s  interest. 


Contract  Requirements 
Drove  DOD’s  Selection  of 
Technical  and  Past 
Performance  Evaluation 
Factors 


The  FAR  and  DOD  guidance  generally  provide  acquisition  staff  flexibility 
to  develop  evaluation  factors  that  meet  their  procurement  needs  and  does 
not  indicate  which  evaluation  factors  should  be  most  important.  The  FAR 
requires  that  DOD  officials  consider,  among  other  things,  past 
performance  on  all  negotiated  competitive  acquisitions  exceeding 
$100,000,  but  DOD  officials  have  broad  discretion  in  selecting  other  non¬ 
cost  factors  and  their  relative  importance.  The  factors  are  intended  to 
provide  meaningful  discriminators  to  evaluate  proposals.  Army,  Navy,  and 
Air  Force  officials  told  us  that  they  formed  interdisciplinary  teams  that 
developed  evaluation  factors  and  the  factors’  relative  importance  by 
consensus. 


We  found  that  88  of  the  129  contracts  we  reviewed  used  a  best  value 
tradeoff  process.  Our  analysis  shows  that  DOD  considered  past 
performance  and  technical  evaluation  factors  as  the  most  important 
among  the  non-cost  factors.  Figure  4  shows  the  five  most  frequently  used 
non-cost  evaluation  factors  for  the  88  contracts  in  our  review  in  which  a 
tradeoff  was  conducted  and  how  often  the  technical  and  past  performance 
factors  were  most  important  among  the  non-cost  factors. 
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Figure  4:  Most  Frequently  Selected  Non-cost  Evaluation  Factors  and  Their  Relative  Importance  in  88  Contracts  Using  a 
Tradeoff  Process 
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Source:  GAO  analysis  of  DOD  contract  documents. 


Note:  For  the  purposes  of  this  report,  in  cases  where  muitipie  non-cost  factors  were  considered  to  be 
of  equai  importance,  GAO  considered  aii  of  these  factors  to  be  most  important.  The  smaii  business 
factors  typicaiiy  consisted  of  evaiuating  the  offerors'  proposed  use  of  smaii  businesses. 


DOD  officials  told  us  that  the  selection  of  these  evaluation  factors  and 
their  relative  importance  was  based  on  specific  acquisition  requirements, 
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such  as  the  ability  to  meet  production  deadlines,  ensure  compatibility  with 
existing  ship  and  aircraft  systems,  or  provide  needed  security  for  dehvery 
of  goods  in  war  zones.  Our  analysis  found  that  DOD  considered  non-cost 
factors  more  important  than  price  in  60  of  the  88  contracts  awarded  using 
a  tradeoff  process.  The  following  illustrate  instances  where  DOD’s 
acquisition  needs  led  them  to  make  non-cost  factors  the  principle  criteria 
for  source  selection. 

•  Army  officials  had  to  quickly  meet  surge  requirements  based  on  a  Joint 
Urgent  Operational  Needs  Statement^'*  for  roadside  bomb  detectors  as  well 
as  services  to  provide  training  and  support  the  system  once  fielded,  and 
accordingly,  made  technical  capability  the  most  important  evaluation 
factor.  The  acquisition  plan  specified  that  deliveries  of  the  critical 
technology  and  support  services  needed  to  be  made  within  6  months  of 
contract  award. 

•  Army  officials  sought  contractors  with  innovative  approaches  and  a 
superior  understanding  on  how  to  counter  the  threat  of  roadside  bombs  in 
awarding  a  professional  services  contract  for  a  range  of  training  programs 
to  be  used  within  the  Mihtary  Service  Combat  Training  Centers.  According 
to  the  Army,  the  selected  contractor  provided  a  proposal  that  was  superior 
in  nearly  all  the  technical  categories  sought  by  the  Army. 

•  The  Navy  considered  contractors’  proposed  technical  approach  the  most 
important  evaluation  factor  for  a  helicopter  upgrade  kit  procurement 
because  the  design  had  to  be  compatible  with  existing  helicopters.  In  this 
case,  the  timing  of  fleet  deployment  was  also  critical  and  the  Navy  sought 
a  contractor  that  could  meet  their  schedule. 

•  DLA  used  a  tradeoff  process  primarily  for  commercial  fuel  contracts  in 
dangerous  areas,  such  as  Iraq  and  Afghanistan,  due  to  the  heightened  need 
for  contractor  reliability  in  these  war  zones.  In  these  situations,  DLA 
officials  explained  the  tradeoff  process  allowed  them  to  emphasize 
secmity  and  past  performance  in  their  evaluations  to  mitigate  acquisition 
risks,  especially  since  they  do  not  know  the  vendors  well. 

•  Army  Corps  of  Engineers  officials  that  needed  to  procure  construction 
services  for  barracks  for  wounded  soldiers  made  the  technical  and 
performance  capability  factors  most  important  because  they  needed  to  be 
responsive  to  new  schedule  and  price  targets.  These  officials  used  the 
tradeoff  process  to  incentivize  timeliness  and  price  reductions,  and  they 
were  also  able  to  obtain  better  features,  such  as  more  durable  materials. 


For  additional  information  on  joint  urgent  operational  needs,  see  GAO,  Warfighter 
Support:  Improvements  to  DOD’s  Urgent  Needs  Processes  Would  Enhance  Oversight  and 
Expedite  Efforts  to  Meet  Critical  Warfighter  Needs,  GAO-10-460  (Washington,  D.C.:  Apr. 
30,  2010). 
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DOD  officials  also  told  us  they  used  these  non-cost  factors  to  encourage 
contractors  to  provide  innovative  solutions  to  meet  DOD’s  needs.  For 
example,  Army  officials  expressed  a  need  for  technological  innovation  in  a 
solicitation  for  equipment,  field  support  services,  and  associated 
maintenance  needed  to  intercept  enemy  communications.  The  Army 
encouraged  the  contractors  to  develop  a  system  that  would  enable  them  to 
upgrade  the  equipment  frequently  over  the  life  of  the  contract.  The 
statement  of  work  clarified  that  these  upgrades  would  be  essential  to 
maintain  relevancy  in  the  battlefield  and  keep  pace  with  technology 
advancements.  Similarly,  Marine  Corps  officials  we  spoke  to  about  an 
urban  warfare  training  system  told  us  that  they  used  the  tradeoff  process 
to  seek  irmovative  designs  when  awarding  a  5-year,  $1  billion  dollar 
contract.  Marine  Corps  officials  indicated  that  they  had  a  system  that 
worked,  but  wanted  to  push  industry  to  come  up  with  a  solution  that 
allowed  the  Marines  to  reconfigure  building  structures  more  quickly  and 
to  provide  more  realistic  and  current  combat  scenarios  prior  to 
deployment.  In  the  winning  design,  the  offeror  proposed  using  modular 
building  sets  that  Marines  could  assemble  more  quickly  to  maximize  the 
training  opportunities  available  in  the  field. 

In  contrast,  our  analysis  found  that  the  28  cases  in  which  DOD  officials 
considered  non-cost  factors  as  equal  to  or  less  important  than  price  were 
nearly  all  related  to  construction  projects.  For  example,  in  15  cases  we 
reviewed,  the  Army  Corps  of  Engineers  considered  non-cost  factors  such 
as  management  and  technical,  experience,  and  past  performance  as  equal 
to  price  to  address  less  complex  project  requirements  such  as  building  a 
new  runway  for  aircraft  and  constructing  a  maintenance  facility.  In  these 
instances,  the  contracting  and  program  officials  were  able  to  request  and 
review  information  from  potential  contractors  and  conduct  a  tradeoff 
process  that  would  not  be  available  through  an  LPTA  approach,  but  still 
considered  price  of  equal  importance  to  non-cost  factors  in  the  award 
decision. 
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DOD  Paid  Relatively  Few 
Price  Differentials  Despite 
Best  Value  Tradeoff 
Solicitations  That 
Emphasized  Non-Cost 
Factors 


For  the  88  contracts  awarded  using  a  best  value  tradeoff  process,  DOD 
considered  whether  to  pay  a  price  differential  in  68  contracts.  Our 
analysis  indicated  that  DOD  selected  the  lower  priced  option  nearly  as 
often  as  it  selected  the  highest  rated,  but  more  costly,  proposal.  In  the  18 
cases  in  which  DOD  officials  decided  not  to  pay  a  price  differential,  they 
determined  that  the  lower  price  outweighed  the  advantages  of  the  offeror 
with  the  higher  technical  rating.  In  doing  so,  DOD  officials  decided  not  to 
pay  over  $800  million  in  price  differentials.  In  29  other  cases,  DOD 
awarded  contracts  to  the  offerors  that  had  both  the  lowest  price  and  the 
highest  non-cost  factor  rating. 


DOD  accepted  a  higher  price  in  21  of  the  68  contracts  in  which  a  price 
differential  was  considered,  for  a  combined  difference  of  more  than 
$230  million.  Most  differentials  paid  were  less  than  5  percent  above  the 
price  submitted  by  the  offeror  next  in  line  for  award.  The  largest  price 
differential  from  the  contracts  in  our  sample  was  48  percent  higher,  or 
roughly  $13.6  million  more,  than  the  next  in  line  offeror’s  price.  In  this 
case.  Marine  Corps  officials  determined  that  the  product — ^burn  resistant 
clothing  for  use  by  soldiers  in  Iraq — was  worth  the  price  difference 
because  it  provided  substantially  greater  2nd  and  3rd  degree  burn 
protection  than  the  product  proposed  by  the  other  offeror.  Figure  5  shows 
the  frequency  with  which  DOD  elected  to  pay  or  not  pay  a  price 
differential  for  the  68  contracts  in  which  a  price  differential  was 
considered,  as  well  as  the  value  of  the  price  differentials  either  paid  or  not 
paid. 


In  the  other  20  contracts,  DOD  did  not  consider  a  price  differential  because  only  one 
vendor  was  considered  for  award  or  these  involved  a  multiple  award  contract. 
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Figure  5:  Price  Differentials  in  the  68  DOD  Contracts  Reviewed  in  Which  a  Tradeoff  Analysis  Was  Conducted 


Contracts  with  a  price  differential  paid 


Contracts  with  no  price  differential  paid 


Total 


Proposals  selected  had  lowest  price  Lower  priced  proposals  selected  outweighed 
and  highest  non-cost  ratings  advantages  of  higher  non-cost  ratings 


21 


29 


18 


Value  of  price  differentials 
as  a  percentage 
of  the  contract  value 

Less  than  or  equal  to  5% 

5%  to  20% 

Greater  than  20% 


1 

6 

11 


68 


Source:  GAO  analysis  of  DOD  contract  documents. 


DOD  Relies  on  the 
Judgment  of  Contracting 
and  Program  Officials  to 
Determine  Whether  Use  of 
Best  Value  Tradeoffs  Meet 
DOD’s  Needs 


DOD  contracting  and  program  officials  believed  that  the  use  of  best  value 
tradeoffs  provide  DOD  an  essential  tool,  which  allows  them  to  obtain 
better  insights  into  the  contractors’  capabilities  and  their  imderstanding  of 
the  government’s  needs,  and  the  reasonableness  of  the  contractor’s 
approach.  DOD  and  military  department  officials  stated  that  they  do  not 
specifically  track  use  of  the  tradeoff  process  to  determine  if  DOD’s 
interests  are  met.  Instead,  they  rely  on  the  judgment  of  contracting  and 
program  officials  to  select  the  best  acquisition  approach  suited  to  program 
requirements  on  a  case  by  case  basis.  Further,  DOD  officials  stated  that 
they  do  not  track  whether  the  solicitation  approach  used  correlated  with 
whether  the  contractor  successfully  met  the  terms  of  the  contract  and 
noted  that  many  factors  ultimately  contribute  to  the  success  or  failure  of 
an  individual  acquisition  that  may  not  have  been  foreseeable  when 
awarding  the  contract.  For  example,  DOD  officials  noted  that  DOD  would 
often  use  a  best  value  tradeoff  process  to  award  a  contract  to  develop  a 
major  weapons  system.  As  our  work  has  found,  DOD  often  encounters 
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cost  increases,  schedule  delays,  and  performance  shortfalls  on  its  major 
systems.  “ 


DOD  Faces  Several 
Challenges  in  Using 
the  Best  Value 
Tradeoff  Process 


DOD  officials  acknowledged  several  challenges  in  using  the  best  value 
tradeoff  process  such  as  the  difficulties  in  developing  meaningful 
evaluation  factors,  the  additional  time  investment  needed  to  conduct  best 
value  procurements,  and  the  business  judgment  required  of  acquisition 
staff  when  compared  to  other  acquisition  approaches.  DOD  officials  also 
noted  that  the  complexity  of  the  tradeoff  process  increases  the  risk  of  bid 
protests.  To  help  address  source  selection  challenges,  DOD  is  drafting  a 
source  selection  guide  to  improve  consistency  and  standardize  source 
selection  procedures  for  competitively  awarded  negotiated  procurements. 

DOD  officials  told  us  that  developing  non-cost  factors  that  meaningfully 
discriminate  between  offers  is  a  challenging  part  of  the  tradeoff  process. 
They  noted  that  as  the  complexity  of  the  acquisition  increases,  so  does  the 
need  for  individuals  with  the  expertise  to  help  develop  the  evaluation 
factors.  For  example.  Army  Corps  of  Engineers  officials  told  us  that  the 
contract  for  one  of  the  world’s  largest  flood  pump  stations  required 
experts  with  experience  in  issues  ranging  from  water  flow  management  to 
real  estate  to  develop  evaluation  factors.  Further,  Navy  officials  explained 
that  while  they  often  use  past  performance  as  a  non-cost  discriminator,  it 
can  be  difficult  to  identify  differences  between  contractor  proposals 
because  contractors  often  provide  their  best  performance  examples  and 
the  government  often  lacks  data  to  evaluate  additional  contractor  projects. 
Our  past  work  has  also  identified  govemmentwide  challenges  in  obtaining 
needed  past  performance  information  to  support  contract  award 
decisions.  Further,  the  absence  of  meaningful  non-cost  discriminators 
can  result  in  offerors  receiving  equal  scores  on  the  factors  that  were 
identified  as  being  significantly  more  important  than  price.  As  such,  the 
decision  may  default  simply  to  a  consideration  of  price  alone.  For 
example.  Air  Force  officials  noted  that  they  are  considering  updating 
factors  used  to  award  task  orders  under  the  Contract  Field  Team  contract 
because  contractors  tend  to  receive  the  highest  ratings  for  each  non-cost 
factor  reviewed,  so  price  is  typically  the  only  discriminator. 


GAO,  Defense  Acquisitions:  Assessment  of  Selected  Weapon  Systems,  GAO-10-388SP 
(Washington  D.C.:  March  30,  2010). 

See  GAO,  Federal  Contractors:  Better  Performance  Information  Needed  to  Support 
Agency  Contract  Award  Decisions,  GAO-09-374  (Washington  D.C.:  April  23,  2009). 
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DOD  officials  also  noted  that  using  the  best  value  tradeoff  process  is  often 
far  more  time-consuming  than  other  approaches.  Navy  officials  told  us 
that  the  tradeoff  process  is  administratively  burdensome  and  requires  a 
large  time  investment  from  program  staff,  which  can  make  it  challenging 
to  keep  the  same  acquisition  team  together  for  an  entire  procurement. 
During  our  site  visits,  many  contract  and  program  staff  told  us  that  the 
tradeoff  process  often  takes  between  18  and  24  months.  In  addition,  in 
Afghanistan  and  Iraq,  the  challenges  of  conducting  a  tradeoff  process  have 
contributed  to  decisions  by  the  CENTCOM  Joint  Theater  Support 
Contracting  Command  and  Army  Corps  of  Engineers  to  discourage  its  use. 
For  example,  recent  Army  Corps  of  Engineer  projects  in  Afghanistan  have 
emphasized  using  simpler,  less  complex  designs  or  requirements  that  are 
more  suitable  for  the  use  of  a  lowest  price  technically  acceptable 
approach. 

The  complex  nature  of  the  best  value  tradeoff  process,  including  decisions 
on  whether  to  pay  a  price  differential,  requires  much  greater  business 
judgment  when  compared  to  other  acquisition  approaches.  DOD  officials 
stated  that  making  tradeoff  decisions,  particularly  when  to  pay  a  price 
differential,  is  among  the  most  difficult  aspects  of  the  tradeoff  process, 
which  will  become  more  challenging  with  less  experienced  staff  coming 
into  the  acquisition  workforce.  DOD  officials  indicated  that  DOD  intends 
to  increase  the  size  of  its  contracting  career  field  by  more  than  6,400 
personnel  through  fiscal  year  2015.  With  the  influx  of  new  staff,  many  of 
the  contracting  officers  we  met  with  noted  challenges  in  preparing  staff  to 
conduct  the  tradeoff  process.  For  example,  a  Navy  contracting  officer  told 
us  that  guidance  and  training  only  go  so  far  to  prepare  acquisition  staff  to 
conduct  best  value  tradeoff  procurements.  Instead,  acquisition  staff  need 
to  be  involved  in  a  number  of  best  value  tradeoff  procurements  to  develop 
the  business  judgment  necessary  to  conduct  a  successful  acquisition. 

DOD  officials  stated  that  the  complexity  of  the  tradeoff  process  also 
increases  the  risk  of  bid  protests.  Of  the  88  contracts  we  reviewed  that 
used  a  tradeoff  process,  15  were  the  subject  of  a  bid  protest  to  GAO.  While 
most  of  the  protests  were  denied,  DOD  took  corrective  actions  in  5  cases, 
including  4  cases  in  which  DOD  terminated  the  contract  or  made  a  new 
source  selection  decision  when  it  determined  that  it  failed  to  adhere  to  the 
solicitations’  requirements. 

Some  of  the  services  have  developed  initiatives  to  address  these 
challenges.  For  example,  the  Air  Force  set  up  an  Acquisition  Center  of 
Excellence  (ACE)  at  Tinker  Air  Force  Base,  which  provides  pre-award 
source  selection  assistance  to  contract  and  program  staff.  Air  Force 
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Conclusions 


officials  stated  that  ACE  reviews  the  evaluation  factors  within  individual 
source  selection  plans,  serves  in  an  advisory  capacity  on  source  selection 
teams  and  holds  workshops  for  contracting  officers.  Similarly,  Army 
officials  at  Ft.  Monmouth’s  Communications — Electronics  Command  have 
developed  an  online  business  tool — the  ASSIST  tool — that  shepherds 
contracting  officers  through  the  solicitation  process.  For  example,  the  tool 
provided  a  list  of  steps  that  must  be  completed  for  best  value  tradeoff 
procurements  and  automatically  routes  documents  through  source 
selection  evaluation  boards  and  other  participating  officials  for  review,  as 
required. 

DOD  is  also  drafting  a  departmentwide  source  selection  guide  to  improve 
consistency  and  standardize  source  selection  procedures  for  competitively 
awarded  negotiated  procurements.  Given  the  influx  of  new  acquisition 
staff,  DOD  officials  stated  they  wanted  to  develop  a  more  prescriptive 
guide  for  best  value  procurements.  While  the  DOD  draft  source  selection 
guidance  contains  information  on  various  aspects  of  the  best  value 
process,  such  as  the  source  selection  decision  document,  DOD  officials 
told  us  it  does  not  address  price  differentials.  Numerous  DOD  officials 
underscored  the  importance  of  training  in  the  use  of  the  best  value 
process,  particularly  training  that  addresses  the  tradeoff  decision  that 
acquisition  staff  must  make.  For  example,  one  Army  Corps  of  Engineers 
official  told  us  that  source  selection  officials  would  benefit  by  training  that 
contains  real  life  lessons  on  how  other  officials  have  made  price 
differential  decisions  during  the  tradeoff  process.  Similarly,  Marine  Corps 
and  Army  officials  told  us  that  while  decisions  are  made  on  a  case-by-case 
basis,  informal  rules  of  thumb  regarding  price  differentials  can  come  into 
play  and  indicated  that  additional  guidance  or  training,  especially  case 
studies  or  scenarios,  would  be  helpful. 

The  Defense  Acquisition  University  (DAU)  is  responsible  for  providing 
training  to  the  DOD  acquisition  workforce.  According  to  DAU  officials, 
they  offer  more  than  10  courses  that  contain  elements  of  the  best  value 
tradeoff  process,  but  none  of  the  current  courses  provide  case  studies  or 
scenarios  that  focus  on  reaching  price  differential  decisions  during  source 
selection.  They  noted  that  once  the  new  source  selection  guidance  is 
implemented,  which  is  anticipated  for  January  2011,  they  plan  to  augment 
existing  contracting  courses  to  reflect  the  new  guidance. 


The  best  value  tradeoff  process  underlies  the  vast  majority  of  DOD 
competitively  awarded  contracts,  and  effective  use  of  this  process  hinges 
on  making  sound  tradeoffs  between  price  and  non-cost  factors.  By 
focusing  on  non-cost  factors,  DOD  anticipates  that  it  will  obtain  technical 
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solutions  that  are  innovative  and  address  complex  and  time-sensitive 
program  requirements.  Applying  a  tradeoff  process,  however,  does  not 
guarantee  successful  acquisitions,  nor  is  it  without  other  challenges.  In 
particular,  using  a  tradeoff  process  can  be  more  complex  and  take  more 
time  than  other  source  selection  methods,  and  requires  that  acquisition 
staff  have  proper  guidance,  needed  skills,  and  sound  business  judgment. 
With  the  anticipated  influx  of  more  than  6,400  DOD  contracting  personnel 
over  the  next  few  years,  providing  a  firm  foundation  for  use  of  the  tradeoff 
process  is  essential.  While  DOD  and  the  military  departments  have  taken 
steps  to  improve  source  selection  procedures,  acquisition  personnel  noted 
a  lack  of  training  to  assist  them  in  deciding  whether  or  not  a  price 
differential  is  warranted  when  making  tradeoff  decisions.  For  example, 
while  DOD’s  new  source  selection  guide  provides  insights  on  the  source 
selection  process,  it  is  silent  on  how  to  reach  decisions  on  when  to  pay  a 
price  differential,  as  is  DOD’s  current  training  curriculum.  DOD  has  an 
opportunity  as  it  updates  its  training  curriculum  to  provide  acquisition 
staff  with  better  insights  using  real  life  examples  on  reaching  tradeoff 
decisions.  Taking  this  step  can  help  DOD  minimize  the  risk  of  paying  a 
price  differential  when  not  warranted  or  losing  the  benefit  of  a  technically 
superior  solution. 


Recommendation  for 
Executive  Action 


To  help  DOD  effectively  employ  the  best  value  tradeoff  process,  we 
recommend  that  the  Secretary  of  Defense  direct  the  Director  of  Defense 
Procurement  and  Acquisition  Policy  to  work  with  the  Defense  Acquisition 
University  to  develop  training  elements,  such  as  case  studies  or  scenarios 
that  focus  on  reaching  tradeoff  decisions,  including  consideration  of  price 
differentials,  as  it  updates  the  source  selection  curriculum. 


Agency  Comments 
and  Our  Evaluation 


DOD  provided  written  comments  on  a  draft  of  this  report.  DOD  concurred 
with  our  recommendation  and  intends  to  request  the  Panel  on  Contracting 
Integrity — comprised  of  senior  DOD  leaders  tasked,  in  part,  to  help 
improve  DOD's  performance — to  assist  the  Defense  Acquisition  University 
in  developing  training  case  studies  and  scenarios  that  focus  on  reaching 
tradeoff  decisions.  DOD’s  letter  is  reprinted  in  appendix  II. 


We  are  sending  copies  of  this  report  to  interested  congressional 
committees  and  the  Secretary  of  Defense.  In  addition,  this  report  will  be 
available  at  no  charge  on  GAO’s  Web  site  at  http://www.gao.gov. 
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Should  you  or  your  staff  have  any  questions  on  the  matters  covered  in  this 
report,  please  contact  me  at  (202)  512-4841  orhuttonj@gao.gov.  Contact 
points  for  our  Offices  of  Congressional  Relations  and  Public  Affairs  may 
be  found  on  the  last  page  of  this  report.  GAO  staff  who  made  major 
contributions  to  this  report  are  listed  in  appendix  III. 

John  P.  Hutton,  Director 
Acquisition  and  Sourcing  Management 
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Appendix  I:  Scope  and  Methodology 


Section  845  of  the  National  Defense  Authorization  Act  for  Fiscal  Year  2010 
directed  GAO  to  report  on  the  Department  of  Defense’s  (DOD)  use  of  the 
best  value  tradeoff  process,  and  specifically  for  cases  in  which  DOD 
evaluated  contractors’  proposals  on  factors  other  than  cost  or  price,  if 
these  non-cost  factors,  when  combined,  were  considered  more  important 
than  cost  or  price/  To  respond  to  the  mandate,  we  determined  (1)  how 
often  and  for  what  types  of  contracts  DOD  used  the  best  value  tradeoff 
process;  (2)  why  and  how  DOD  used  the  best  value  tradeoff  process;  and 
(3)  what  challenges,  if  any,  DOD  faces  in  using  the  best  value  tradeoff 
process. 

To  determine  how  often  and  for  what  types  of  contracts  DOD  used  the 
best  value  tradeoff  process,  we  used  data  from  the  Federal  Procurement 
Data  System-Next  Generation  (FPDS-NG)  as  of  January  2010  to  identify  a 
population  of  contracts  based  on  the  following  criteria:  (1)  newly  awarded 
by  DOD  in  fiscal  year  2009;  (2)  competitively  awarded,  and  (3)  had 
obligations  of  $25  million  or  more  in  fiscal  year  2009. 

We  established  the  $25  million  threshold  because  the  Defense  Federal 
Acquisition  Regulation  Supplement  (DFARS)  requires  contracts  with  total 
estimated  costs  of  $25  million  or  more  in  any  fiscal  year  to  prepare  written 
acquisition  plans,  which  contain  information  about  the  source  selection 
approach.  This  analysis  identified  363  contracts.  From  this  population,  we 
selected  a  probability  sample  of  160  contracts,  including  60  indefinite 
delivery  contracts,  and  reviewed  associated  solicitations,  source  selection 
decision  documents,  and  other  contract  documents  to  determine  the 
solicitation  approach  DOD  used. 

We  verified  the  obligations  and  contract  award  fields  in  FPDS-NG  with 
contract  data  to  ensure  that  the  contracts  within  our  sample  were  within 
scope.  Thirty-one  contracts  from  our  initial  sample  of  160  contracts  were 
outside  the  scope  of  our  review  because  they  were  incorrectly  coded  in 
key  parameters,  such  as  being  coded  as  competitively  awarded  when  they 
were  not  or  had  misreported  the  amount  of  obligations  made  on  the 
contract  or  task  order.  We  excluded  these  contracts  from  our  sample  and 
determined  that  FPDS-NG  was  sufficiently  reliable  for  the  purposes  of  our 
review  after  adjusting  for  these  errors. 


^  For  the  purposes  of  this  report,  we  use  the  term  “non-cost  factors”  for  both  non-cost  and 
non-price  evaluation  factors  and  the  term  “price”  for  both  cost  and  price,  despite  the 
different  ways  that  they  are  evaluated  under  the  FAR. 
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Because  we  followed  a  probability  procedure  based  on  random  selections, 
our  sample  is  only  one  of  a  large  number  of  samples  that  we  might  have 
drawn.  Since  each  sample  could  have  provided  different  estimates,  we 
express  our  confidence  in  the  precision  of  our  particular  sample’s  results 
as  a  95  percent  confidence  interval  (e.g.,  plus  or  minus  8  percentage 
points).  This  is  the  interval  that  would  contain  the  actual  population  value 
for  95  percent  of  the  samples  we  could  have  drawn.  Unless  otherwise 
noted,  percentage  estimates  based  on  our  sample  have  95  percent 
confidence  intervals  that  are  within  plus  or  minus  8  percentage  points  of 
the  estimate  itself.  Confidence  intervals  for  other  numeric  estimates  are 
reported  along  with  the  estimate  itself  Table  1  summarizes  the  estimated 
percentage  of  contracts  of  various  source  selection  approaches  reviewed. 


Table  1 :  Estimated  Percentages  of  Source  Selection  Approaches  Reviewed 

Approach 

Estimated  percent 
representation 

Best  value 

Tradeoff  process 

55% 

Lowest  price  technically  acceptable 

21% 

Sealed  bid 

4% 

Erroroneously  reported  as  being  within  our  scope  of  review 

19% 

Source:  GAO  analysis  of  DOD  contract  information. 


Note:  Percentages  do  notsum  to  100  percent  due  to  rounding. 

Based  on  our  analysis  of  the  remaining  129  sample  contracts,  we  estimate 
that  the  total  number  of  best  value  tradeoff,  lowest  price  technically 
acceptable,  or  sealed  bid  award  decisions  (in-scope  contracts)  in  the  full 
population  of  interest  was  about  293.^  For  contracts  that  utilized  a  best 
value  tradeoff  process,  we  categorized  them  based  on  the  relative 
importance  placed  on  price.  In  addition,  we  determined  contract  type,  the 
type  of  procurement  (product  versus  services),  and  the  type  of  product  or 
service  for  our  sample  contracts  using  FPDS-NG  data  and  verified  this 
information  with  the  contract  documents. 

To  determine  why  and  how  DOD  used  the  best  value  tradeoff  process  and, 
in  particular,  when  non-cost  factors  were  considered  more  important  than 
price,  we  obtained  and  reviewed  DOD  and  service  level  acquisition 


^  The  95  percent  confidence  interval  for  this  estimate  is  from  276  to  309  contracts. 
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guidance  related  to  source  selection  policies  and  procedures  that  describe 
how  and  when  the  tradeoff  process  may  be  used,  including  those  used  for 
issuing  task  orders.  In  addition,  for  each  of  the  contracts  within  our 
sample,  we  obtained  contract  documentation  including  the  acquisition 
plan,  solicitation,  and  source  selection  decision  memorandum  and 
reviewed  them  in  preparation  for  interviews  with  DOD  officials.  In  several 
cases,  the  solicitation  was  unclear  as  to  which  type  of  tradeoff  process 
was  used.  In  these  cases,  we  relied  on  the  source  selection  decision 
document  to  categorize  the  tradeoff  process  used. 

We  judgmentally  selected  buying  activities  to  visit  based  on  factors 
including  the  number  of  contracts  awarded  on  a  best  value  tradeoff  basis, 
contract  type,  and  goods  or  services  procured.  Buying  activities  included 
at  least  one  command  from  each  military  department  as  well  as  a  defense 
agency.  We  reviewed  27  contracts  and  23  task  orders  through  our  site 
visits.  Specifically,  we  judgmentally  selected  23  of  48  task  orders  for 
review  by  compiling  all  task  orders  issued  on  indefinite  delivery/indefmite 
quantity  contracts  obligating  over  $10  million  that  were  administered  by 
the  officials  at  the  sites  we  visited.  We  chose  this  dollar  threshold  to 
exceed  the  FAR  requirement  to  provide  fair  opportimity  notices  for  task 
orders  valued  at  $5  million  or  more.  Results  from  these  selected  contracts 
or  task  orders  cannot  be  generalized  beyond  the  specific  contract 
contracts  or  task  orders  selected. 

During  the  course  of  our  review,  we  interviewed  officials  from  the 
following  commands: 

•  Department  of  the  Army,  U.S.  Army  Corps  of  Engineers,  New  Orleans 
District  Office,  Louisiana,  and  Afghanistan  Engineering  District,  Kabul  and 
Kandahar,  Afghanistan; 

•  Department  of  the  Army,  Armament  Research,  Development  and 
Engineering  Center,  Picatinny  Arsenal,  New  Jersey; 

•  Department  of  the  Army,  Communications-Electronics  Command,  Fort 
Monmouth,  New  Jersey; 

•  Department  of  the  Navy,  Marine  Corps  Combat  Development  Command, 
Quantico,  Virginia; 

•  Department  of  the  Navy,  Naval  Air  Systems  Command,  Patuxent  River, 
Maryland  and  Lakehurst,  New  Jersey; 

•  Department  of  the  Air  Force,  Air  Force  Materiel  Command,  Tinker  Air 
Force  Base,  Oklahoma; 

•  Defense  Logistics  Agency  Energy,  Ft.  Belvoir,  Virginia;  and 

•  Joint  Theater  Support  Contracting  Command,  U.S.  Central  Command, 
Kabul  and  Kandahar,  Afghanistan,  and  Baghdad,  Iraq. 
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Appendix  I:  Scope  and  Methodology 


We  interviewed  DOD  acquisition  and  contracting  officials  to  identify  their 
rationale  for  the  selected  source  selection  approach  (e.g.,  the  thought 
process  behind  why  a  best  value  approach  was  chosen  over  other 
approaches).  For  award  decisions  that  used  a  best  value  tradeoff  process, 
we  discussed  why  the  evaluation  factors  were  chosen  and  how  their 
relative  weights  were  assigned.  We  also  interviewed  officials  about  the 
process  used  and  the  underlying  rationale  when  issuing  selected  task 
orders.  We  also  interviewed  officials  to  determine  what  the  expected 
outcomes  were  from  using  the  best  value  tradeoff  process. 

We  reviewed  applicable  DOD  source  selection  decision  documents  and 
related  memoranda  to  determine  how  often  DOD  paid  a  price  differential, 
the  amount  of  the  price  differential,  and  the  reasons  that  were  given 
underlying  the  decision  to  pay  a  higher  price.  We  defined  a  price 
differential  as  a  positive  difference  in  price  between  the  offeror  who 
received  the  award  and  the  offeror  next  in  line  for  award. 

To  determine  what  challenges  if  any,  DOD  faces  in  using  the  best  value 
tradeoff  process,  we  reviewed  DOD  guidance  and  interviewed  officials 
from  Defense  Procurement  and  Acquisition  Policy,  the  military 
departments  and  defense  agencies. 

We  conducted  this  performance  audit  from  March  2010  through  October 
2010  in  accordance  with  generally  accepted  government  auditing 
standards.  Those  standards  require  that  we  plan  and  perform  the  audit  to 
obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe 
that  the  evidence  obtained  provides  a  reasonable  basis  for  our  findings 
and  conclusions  based  on  our  audit  objectives. 
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Appendix  II:  Comments  from  the  Department 
of  Defense 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

3000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3000 


ACQUISITION, 
TECHNOLOGY 
AND  LOGISTICS 


Mr.  John  Hutton  OCT  2  0  2010 

Director,  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  N.W. 

Washington,  DC  20548 

Dear  Mr.  Hutton: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  draft  report  11-8 
“DEFENSE  CONTRACTING:  Enhanced  Training  Could  Strengthen  DoD’s  Best  Value 
Tradeoff  Decisions,”  dated  October  2010,  (GAO  Code  120890). 

The  Department  has  reviewed  the  draft  report  and  concurs  with  its  recommendation.  I 
intend  to  request  the  Panel  on  Contracting  Integrity,  Subcommittee  5,  Appropriate  Contracting 
Approaches  and  Techniques,  to  assist  the  Defense  Acquisition  University  in  developing  the 
training  case  studies  and  scenarios  that  you  recommended.  I  thank  you  for  the  opportunity  to 
comment  on  the  draft  report. 


and  Acquisition  Policy 
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Appendix  III:  GAO  Contact  and  Staff 
Acknowledgments 


GAO  Contact 


John  P.  Hutton,  (202)  512-4841  orhuttonj@gao.gov 


Acknowledgments 


In  addition  to  the  contact  named  above,  Timothy  DiNapoli,  Assistant 
Director;  William  Russell,  Katheryn  Hubbell,  Paige  Muegenburg,  Jodi 
Munson,  Anna  Russell,  Sylvia  Schatz,  Roxanna  Sun,  and  Bob  Swierczek 
made  key  contributions  to  this  report. 


(120890) 
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GAO’S  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and  policies; 
and  provides  analyses,  recommendations,  and  other  assistance  to  help 
Congress  make  informed  oversight,  policy,  and  funding  decisions.  GAO’s 
commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost 
is  through  GAO’s  Web  site  (www.gao.gov).  Each  weekday  afternoon,  GAO 
posts  on  its  Web  site  newly  released  reports,  testimony,  and 
correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted  products, 
go  to  www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of  production 
and  distribution  and  depends  on  the  number  of  pages  in  the  publication 
and  whether  the  publication  is  printed  in  color  or  black  and  white.  Pricing 
and  ordering  irrformation  is  posted  on  GAO’s  Web  site, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 

MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Ralph  Dawn,  Managing  Director,  dawnr@gao.gov,  (202)  512-4400 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7125 
Washington,  DC  20548 

Public  Affairs 


Chuck  Young,  Managing  Director,  youngcl@gao.gov,  (202)  512-4800 
U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 
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